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ARTIFICIAL INTELLIGENCE 

This report should be read in conjunction with last year's.* This is 
particularly important to obtain a rounded view of our work on vi- 
sion. In fact, much of what we say thia year is logically prior to 
much that we said last year. Thus last year we discussed the ab- 
stract and theoretical problems related to the interpretation of pic- 
tures presented in a clean form (as line drawings or as subsets of 
an abatract retina) while this year wo say more about how to obtain 
such pictures 1mm the real world. The reason of. course lies in 
the fact that the ft higher lever work did not depend as heavily on 
prior solutions ol hardware and systems problems. 

Wc have not reported new work that lies directly in the line of de- 
velopment of ideas discussed Last year* In partieulnTj, we have deep- 
ened and generalized some of the theorems on computational geom- 
etry. But the new state of knowledge is indistinguishable from the 
old on the level of discussion of this kind of report Similarly we 
have not reported very new work which is still at a too primitive 
level of development to be presented Intelligibly. This includes work 
on natural language processing, concept Information, teaching and a 
number of mathematical topics. The time constant for a project of 
this sort is longer than a year. These matters will be dealt with 
Ln a further report due In the Fall ol 1970. 

Analysis of Visual Scenes: The Concept of Vertical Problem -S olyi ng 

One can use vision In many ways to find out about objects in space. 
Let us begin with some of the simplest ideas, and then move on to 
the deeper problems. IT. the camera can more, one could use stereo- 
scopic correlations of two pictures. Or one might use local opera- 
tions to measure motion parallax. These do not solve all problems. 
Stereoscopy does not work well on featureless surfaces or curved 
boundaries; motion effects are misleading on plain or shiny objects. 
Nevertheless, both methods can help find the three-dimensional loca- 
tions of parts of visual objects. 

Focusing provides a third method of location. Its distinctive feature 
is that it needs only a single eye in a Ji*ed location, provided that 
the eye has a wide enough optic aperture* For then the idea of 
measuring distance by stereoscopic parallax merges with that of 
measuring distance by finding the be&t focus setting. Figure 1 shows 
two views of an object lying on a plane; in the photograph it le 
hard to sec the object because of camouflage * (The machine sees 
from a little below the viewpoint of the left picture.) Berthold K. P. 
Horn has developed a program that can iind the lens setting for best 
focus at each point In the visual Held. The procedure, applied to a 
series of points along a horizontal scan through the middle of the 

• See p. 43 r 



cube, yields the profile shown Ln Fig. 2. fThe location of the back- 
ground is less definite than that oJ the cube because of th& back- 
ground'^ obliquity to the- earner*. J Horn's program uses local Fou- 
rier transforms and compares the relative energy in the high and 
low spatial frequencies. It servo-controls the lens to maximize the 
highs, and it focuses at least as well as one can no manually. 
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Hum will discuss the focusing procedure in detail in his thesis 
''Shape Irom Hhadinij;: A Method fur Finding the Shape of a Smooth, 
OpsMnie Object from One View*. Two techniques are available; one 



uses a last circular scan to obtain a periodic function characteristic 3 

of a large neighborhood; the other makes a faster scan of a smaller 
square. An operation manual (Horn, "Focusing', Aj r Memo ISO) gives 
instructions for using the system with the PDP-6 computer and its 
optical-mechanical accessories, as well as some of. the theory of 
the focusing procedure. 

Now we have described three methods for optical range -finding. 
There arc many other ways to attack just this one aspect of visual 
technology — such as flying-spot a canning from a site off-center 
from the camera, holographic methods, and even optical radar, We 
must net, however, allow the myriad of technological possibilities 
to divert ug from the deeper — and incompletely understood — prob- 
lem of developing a visual system flexible enough to deal with real- 
world problems. The goal we have set ourselves is to find how to 
make a system that can approach the versatility of human vision* 
One outstanding feature of that system is its "pass Lvcness * -- the 
great extent to which it can see without much special preparation 
or interaction with the objects in the scene* The secret lies, in the 
intelligent viewer's ability to combine what he sees with what he 
knows about his world. 

To pursue this, we have concentrated on the problem of reconstruct- 
ing a three -dimensional structure using only a single monocular pic- 
ture, People are quite good at understanding what is shown in a 
photograph; we should like to know how to make a machine do this* 
Now, with a very few exceptions, the many past attempts at com- 
puter analysis of scenes have been rather fruitless, and we should 
try to understand what went wrong. Almost all of those pa3t at- 
tempts followed the same general plan: the picture is subjected to 
a sequence of transformations; oach transformation is intended,, in 
turn, to produce a successively more abstract ru presentation until, 
finally, one obtains the desired description of the scene. Typically* 
such a seqjuenee might be: 

1) Remove noise (by clipping, smoothing, etc.); 

2) Enhance features (by boosting gradients, etc.); 
3J Extract features (finding edges, vertices, etc,); 

4) Group features into objects (by regions, parallelisms, etc.); 

5) Identify objects (by partial matches, etc*}* 

Although there is a great deal of plausibility to this idea of pro- 
gressing relentlessly from local to global, the concept of serial 
stages of pre-processing does not actually work well In practice. 

It is simply not suited to the real problem. Errors and assumptions 
made at each level are passed on to the next, and even if each 



is quite clover at hgw it handles its data, the accumulation of mis- 
takes over many stages leads to chaotic over- all results. The basic 
urammar of the problem is too context -depuiLuent, The appearance of 
an object' 5 features (and even their occurrence or non -occurrence) 
usually depends on global aspects of the arrangement ami illumina- 
tion of the scene. One must cope, for example, with 

1) Direct line -of -sight occlusion of parts of objects , 

2) Shadow occlusions that depend on the directions of lighting, 
Z] Highlights, 

4) Reflections, 

5) Textures, 

6) Decorations , 

7) Many other interactions between visual features and 
Spatial for my r 

Accordingly, the inevitable ambiguity problems met at each level -- 
"Is this an edge or not?* or ^Are these two features part of the 
same object?* ~ are often not solvable at that level, One can se- 
lect a plausible interpretation only tty using a wider variety of 
knowledge about the real world -- knowledge that ranges from prin- 
ciples of optics and geometry to knowledge about the particular en- 
vironment and the objects likely to be In it + 

In the traditional processing sequence outlined above --we shall 
call li the horizontal vision system -- different kinds of knowledge 
are Implicit at each leveL The principles of optics are involved 
because each visual point represents a distribution function of space 
points in a way thai depends on focus, scattering, reflection and 
noise. In the extraction of features, the processor must know wheth- 
er the objects are likely to have straight edges, or tewture boun- 
daries, or polished surfaces. In analyzing a room, to give an ex- 
treme but real example., imagine the resulting chaos if the system 
did not know the significance of a picture frame? 

At the level of grouping features and identify fng spatial bodies, we 
have already seen (A-L Group, P. R. V; page 43.) how problems of 
projections and occlusion of three-dimensional objects lead us 
away from the simple template-matching schemes thai work fairly 
well for two-dimensional problems. We found, however, that many 
of these difficulties could be handled by symbolic-description sys- 
tems, and we shall assume that the reader Is familiar with ProL 
Adolfo Guzman's work, either thruu&jh the survey in Progress Re- 
port V, or through his Doctoral dissertation. We now conclude that 
the lower-level aspects uf vision, too, are best treated as problems 



In artificial intelligence to be handled by a mixture of general meth- 
ods and special knowledge. Because the different kinds of knowledge 
interact at different levels, we must provide channels tor such inter- 
actions so that hypotheses about high-level things like objects ~- 
perhaps proposed by heuristics that use local evidence -- can be 
confirmedj rejected or revised by returning to other levels for other 
kinds of evidence. We use the term vertical system for this type of 
organization* 

We have not yet enough experience with vertically to discuss it ab- 
stractly. But we now know a substantial amount about its application 
to visual problems; the body of this section reports what we have 
found so far. 

Optical Anatomy of a Sim ple Scene 

In Fig. 3 we see three cubes with dull white painted surfaces against 
a dark background, illuminated by concentrated light from a lamp 
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above and to the right of the camera. The data and methods used 
here arc from work by Arnold K. Griffith. Figure 4 shows three 
plots of the light intensity, measured along three horizontal scans, 
each of 1000 points across the picture* 

It Is difficult to discern very much in these plots. The [text illus- 
tration, Fig, 5, shows the result of applying, to * sequence of such 
cross sections, a kind of second-derivative operation averaged over 
enough adjacent sample points to giye flat plots over regions that 
have reasonably uniform gradients, (The photograph and the meas- 
urements were taken from slightly different positions,} 




We have marked a number of typical features: 

a is an outer edge. Against the dark background, it Is a sim- 
ple, suddu:: c::]aji6ii in intent ity, giving a typical "bipolar'" 
pulse in the smoothed second derivative. 

b ia a more symmetrical feature; it is due to the "highlight" 
reflection on the front edge of the lower right cube. 

b T Is a superposition of the effects of a b and a small a. 



c Is the reflection of the bright surface of the upper cube in 
the top of the lower right cube* Although the paint there is 
dull, this surface is seen at a low angle, and tills enhances 

reflections , 

d is the crack between the lower cubea. The brightnuya 
measurements are all logarithmic, and truncated so that the 
plots wOnH overlap. 

e Lb the spot yf dirt on the upper front corner Of the lower 
right cube. The edges and corners of objects often have high- 
lights and often are dirty. 

f is a shadow boundary visible on the dark background. 

f is another shadow boundary f somewhat less sharp because 

of penumbra and depth of locus* 

The internal edges, such as at b p of uniformly colored objects often 

give small signals. Their width, In our plots, is an illusion duo to 
the smoothing operation; on sharp corners* the highlight line is 
very narrow and easily missed by a coarse scan, 

"The history of attempts to write edge- finding- and edge -following 
programs is long and inconclusive. Because the results were so 
obscure, Annette Herskovits {AI Memo IB 3) and Griffith made sepa- 
rate studies of the edgea of geometric at objects; both concluded 
that the most common phenomena were superpositions of three ef- 
fects (see Fig + 6). 



(1) a Simple Step 



(2) a slope change 



J\ . . (5} a highlight or 

1 i a crack 



Fig. e 



They both investigated various detection filter methods (or these* 
Griffith's thesis will include a theory of optical detection of edges 
under various assumptions about their intrinsic character and about 
the kinds uf noise one might expect in a vision system, The most 
sensitive methods for detecting edges use two-dimensional operations, 
but these are very expensive with conventional hardware* because of 
the large amount of high- resolution information. 

When a compromise must be made, it in not especially good simply 
to use a coaxser homogeneous scan; for instead of the arrangement 
of Fig. 7 j one can use that of Fig. 3, which has the same mean den- 
sity hut is better at catching thin edges, 




Fig. 7 



Pig. 8 



Now we apply this idea to some more realistic , cluttered scenes. 
We mark with short dashes the local maxima of the output of the 
edge-filter, along a mesh oi fine horizontal and vertical scans. The 
problem remains to convert this set of local features into lines, and 
then into objects. Figureb ft and 10 show the results of a process that 
uses a projection operation sensitive only to straight- line segments. 






Fig. 9 



Although it is not good lor close -packed features, it Is eonservatiYe 
and does not propose many false Hnes + Griffith's thesis will give 
details. 
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Fig. 10 

Another kind of process works in a comple mentaxy manner; it labels 
points whose neighborhoods are relatively homogeneous. Then the 
system finds the boundaries ol the connected regions of such points . 
Thomas O. Rinford (AI Memo 1&2) describes experiments on such a 
system y There are many problems in deciding how to reduce the 
region boundaries to useful line -descriptions. We see In Fig. 13 the 
result of such a system applied to the relatively simple scene shown 
in Fi£, 11, 








Fig. 12 



10 Still another approach to line-fi riding uses a two-dimensional local- 

gradient detector, followed by a scheme for assigning the lor ally 
maximal futures to edges, as in the early work of L. G. Roberts 
(Itef. !)♦ Richard rj. Greenblatt is developing a system of this sort. 

Problems exist at every stage of such processes, At each level, 
the selection of relevant features requires some a priori knowledge 
about tlie local world. Our vertically thesis holds that one cannot 
expect any one decision policy to work over a very wide range of 
situations, but that even a little feedback in this selection will help 
considerably. For example, each of the systems mentioned above 
will miss some edges of some objects, because of the problems of 
resolution, illumination, focus, contrast, texture or noise. II the 
system misses an interior edge, the SEE program (or rather, one 
version of it) may have to propose one object in place of two, as 
In Fig, 13, or two obje&ts in place of one. 




Fig. 18 

Assuming we are in a world oi geometrical bodies, there is a varie- 
ty of ways in which to use knowledge of the fact to propose correc- 
tions, (These will have to be verified, but proposing them is most 
of the battle,) 




Fig. 14 



Missed edges, for example, are often related to concavity (&ee 
Fig. 14), and, in a rather Bayeeian way, this suggests a search for 
missed lines radiating from the concave vertices into the figured 
LjiLL-riur, 



11 




Fig. 15 

In Fig. 15, wo indicate proposed lines of several kinds; V directly 
to other vertices; E interior extensions of the vertex r s edges; and 
L an {absolute) vertical edge (very common in re,a) interior scenes}. 
One might further propose parallels and lines that make confocal 
triplets. 

It is remarkable how much can be done with such a line proposer. 
In the ya^e of structures made entirely uf rectangular solids, Prof, 
Manuel Bhijp, showed (Fig. 16.) that a remarkable number of interior 
edges can be reconstructed just from the outer profile of the scene. 




Tig, 16 



12 The number gf lines proposed by such a scheme can be held to the 

order of tens, rather thaw of thousands,, And the cost of verifying the 
eii&tence of an edge with a specified location in enormously smaller 
than that of finding all such features independently, because — for 
the- same statistical confidence -- rejecting a particular null-hypothe- 
sis is always much easier than screening all of a large family of 
possibilities. The use of proposer-verifier system can thus reduce 
the total picture-processing effort by relaxing the tolerances on the 
early, brute-force, feature -finding stages. In his forthcoming thesis, 
Griffith will fjwe details of a complete system, already working,, 
that docs this. A first stage finds some of the edges in the scene. 
Then new lines are proposed on bases of parallelisms, region com- 
pletion, etc*, and verified by the detectors mentioned earlier, 

By using a priori information, one can often get much more out of 
a picture than might seem to be in iL> Assuming (correctly) that 
the sphere in Fig. 17 fa) is uniformly colored and that the light 
qomtjd from a compact source, one of Horn h s programs is able to 
reconstruct the surface by solving the appropriate differential equa- 
tions. Then this program produces the stereoscopic pair of Fig. 17 Co) 
and 17(c), (Some readers wUl be able to fuse these by looking at a 
virtual point beyond the page.) The sharp shadow detail confirms, 
with the picture, the hypothesis of sharp Illumination* Horn will pre- 
sent details of this program in his thesis. The method is quite com- 
plementary to stereoscopy since it relies on uniformity of surface, 
whereas stereoscopy and focusing need inhomogeneous surface detail 
or discontinuities. 
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F*g« 17(a) Fig. 17(b) Fig, 17(c) 

Even if the surface is not of uniform color and texture, there is still 
much more that can be done with a monocular picture, Lawrence J. 
Krakauer is developing a system lhat analyzes scenes such as a 
bowl of fruit. The process begins by locating and analysing tllumina- 
Liun maxima; it appears thai, from their intensity-Shape behavior 
one can. In many cases, distinguish dull from shiny surfaces. Local 
maxima connected by an illuminated band are likely to be on the 
same object, and Krakauer is testing some other heuristics for as- 
sociating highlights with edge* (Fig. 16]. 
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Features 

Alone 



Krakauer's goal is to discover visual characteristics of surface 
properties, and to establish heuristics lor grouping "non-geometric* 
features into natural objects j perhaps as Guzman did lor geometri- 
cal objects. 

The Vision System 

We have discussed Guzman's SEE program (see page 43); this 
program assumes a scene description in terms oi edges , verti- 
ces and regions ¥ and produces a proposed assignment of these fea- 
tures to a set of three- dlniens tonal bodies, Guzman's thesis describes 
in detail a more advanced version of that project. It includes addi- 
tional tieuri sti^s for linking parts of objects, analysis of the system 1 & 
behavlor d discussion of and heuristics for correction of mistakes be- 
cause of pre- processor errors, jmd some analysts of the system er- 
ra.-ii due to inherent ambiguities in ordinary scenes and in a variety 
oi standard "optical illusion' 1 scenes. Guzman 1 s thesis also includes 
some observations about the problem of matching features between 
stereo pairs r In particular t the following simple technique, when it 
is embedded in a vertical system > will solve the majority of such 
problems* Consider two views of a geometrical scene (Fig* 19). in 
any stereo pair, one can dissect the two pictures into sets of match- 
ing line -pairs, defined by the planes through the two eye-points ♦ Any 
physical object visible to both eyes will be sandwiched between the 
same highest and lowest such lines, as suggested by Fig, 2:0. For 
two different objects, it is unlikely this will be true by coincidence, 
in specially if the objects have more than one visible face, since the 
sandwich proposition ia true also for each facel Thus, onee enough 
point features are identified in the monocular pictures, one will have 
little difficulty in matching them between the pictures, and expensive 
cross- correlations should be unnecessary* This matching works well 
even when the two viewpoints are far apart* 
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Fig. 19 




Fig, 20 

In another attack on stereoscopic vteton, Dwvid N. Perkins, Jr. is 

developing a system that compares two views and produces depth in- 
formatipn. His matching procedure is complicated by the requirement 
that It be tolerant of the maj^y kinds of errors that pre-procesaors 
are likely to make. It proceeds by matching vertices and area topo- 
logieally, with some geometric constraints, and with a back-up and 
search procedure for finding maximal matches ol substructures when 
tiiere are possible local ambiguities. Then, given the best topological 
match, the program proceeds to analyse the arcs that are thus pro- 
posed for matching to obtain space curves t For example, the stereo 
pair of Fig + 21 is converted, by Binford's TGPOLQGE5T system., to 
the views of Fig. 32 fa) and 22(b), 

Then Perkins's system, on request, produces views as seen Irom 
the right side and irom above in Fig 4 22(c) and 22(d) (shown hers 
without suppression of lines that might be hidden by surfaces )♦ 
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Fig, 21 





(a) 



(b) 
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Fig. 22 



16 We now have a complete horizontal system of programs connecting 

Binford's topological pre- processor with Guzman's SEE program 
and on through lo Patrick Winston's new system (described later in 
the report) that recognizes some particular types of objects -- e.g., 
wedges, pyramids t and roctatigular blocks — and learns to identify 
some multi -object structure such, as rows, towers and bridges. 

■ 

Professor llosakere N, V. Mahabala developed the TQPOIjOOlST-to- 

SEE interface. This program , SETUP ("Pre- processor tor Programs 
Which Recognise Scenes" t A.I. Memo 177), gobbles a list of line seg- 
ments and produces a complete topological description of the graph 
formed by the lines* Because such features as- lines and vertices 
found by pre -procey aura have some uncertainty In location, there 
are usually serious problems in deciding when two edges are really 
the same, or in connecting edges and localizing vertices. SETUP 
contains heuristics for plausible guesses about such matters. All 
lines are treated as enclosed within strips of a certain width, and 
all probJenis about closure, intersection, containment, eolinearity, 
etc., are resolved by procedures that are based on a single predicate 
about the orientation of a point with respect to one of these half -line 
Strips. Although this may not be any better in performance than 
Other segment- joining and line-grouping criteria, it is probably no 
worse, and Mahabala's system in distinctive in having greater logi- 
cal clarity than any other system we nave seen before. It is there- 
fore muth more likely to be improved by advances in theory I 

Finding the edges themselves is still a problem. When one k^ows, 
a priori , that they are straight, the criteria Griffith and Herskovibi 
developed are probably adequate for practical purposes, and these 
are further subject to substantial heuristic speed-up innovations. 
For the more general problem or describing an ordered set of 
points (such as one obtains as the houndary of a * homogeneous re- 
gion") as a curve, we need a systematic way to apply various hinds 
of a priori knowledge. At present we have a variety of such at- 
tempts, such as POLYSEG (GriMth h A.I. Memo 131), the Greenblatt- 
Holloway line -finder (A.L Memo 101), and three others, by Rinford, 
Greenblatt, and Jayant M. -Shah. Unfortunately, none of these Is well 
enough understood to be considered theoretically firm. A variety ol 
other general-pur posu curve-segmentation procedures has been de- 
scribed Ln the literature. But no one has really come to grips with 
the basic problem of incorporating the relevant a priori information, 
and we conclude that this is one of the aspects to bo faced in de- 
signing the vertical vision System^ 

Richard Orban has developed a new program, ERASER, which de- 
tects and removes shadow boundaries from geometrical scenes, 
ilRAijER resemble^ SEE in that it uses the same classification of 
vertex features, but it also uses information about the relative 



brightness of regions. Its heuristics are baaed largely on the abun- 
dance of L N T and X types of vertices on the boundaries of shadow 
regions. ERASER is designed to critici2e the output of SETUP, to 
remove shadow boundaries, ^nd then to re -submit the result to 
SETL"P before passing the problem on to SEE. II will not remove. 
all shadows,, but it is conservative about not removing real edges. 
Examples of ERASER'S performing- well are given in Ftp. 23. 
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Fig. 23 



The ERASER system also contains some heuristics that Efnford de- 
veloped for guessing the direction ol illumination; this is used to 
bias the operation of ERASER and is available to the rest of the 
system. In geometrical scenes, the system can measure the angles 
between real vertical edges and tliose shadgw boundaries that appear 
to lie on horizontal surfaces in order to get a quantitative measure 
of illumination angle. As an application ol vertically, the operation 
ot both ERASER and SEE could be enhanced by verifying, at a high- 
er level, that some of the shadow boundaries Ue across otherwise 
uniform surfaces (say d by using Perkins's stereo system) or by 
verifying, at a lower level, tlial the proposed Inner shadow bound- 
aries are less than sharp, i.e., have penumbras. 
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Mechanical Structure Analysis of visual S cenes 

Blum and Griffith have written a program that can analyze the- sta- 
bility of the three-dimensional structure of rectangular blocks< The 
urogram uses this analysis in a p lann ing scheme to propose the 
order in which the structure is to be built. Even Ln maniputating 
toy blocks, there are problems; one can ask which of these struc- 
tures can be constructed with one hand. 

Referring to Fig, 24, the One on the left cannot be built, the pro- 
gram asserts, because there is no stable three- bluek sub=part ol 
the structure- But with another interpretation of the rule a, one could 
first assemble the upper three blocks on the floor, then lift them 
into position. We expect our more advanced construction- planning 
programs to be able to use more advanced strategies in which sub- 
assemblies are hq identified. 
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Fig T 24 



Winston is completing a program that learns to recognize types of 
structures from sequences of examples* We consider it to he a 
major advance in the areas generally known as c one-ept-formation , 
or machine - lear ning . Given a scene (represented by a collection of 
regions, as produced at the output of the SEE program) h Winston's 
program attempts to describe the scene in terms of elementary ob- 
jects and already -known sub -structure a and relations, using a de- 
scriptive language reminiscent of that Dr T Thymas G r Evans used 
(Ref. 2), but Winston" 3 language is further developed, 

for example, the scene of Fig. 25 leads to a description like that 
shown to its right. We tell the program this is a picture of an 
ARCti. 

Next, we inform it that the picture of Fig. 26 is not -an- ARCH. The 
program then proceeds to compile a new description {of ARCH), as 
shown to the figure T s right. 

This new description of ARCH is obtained by comparing the descrip- 
tions of the two scunes, thus providing a second- level description. 
The must- not -abut relation is the outstanding difference it discovers, 



and it modifies the description of ARCH to require that the abutting 
relation not hold between the two support Lkg blacks. (Wins ton 1 & pro- 
gram Is equipped ab initio with heuristics lor proposing support and 
mm- supporting contact.) 
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Now,, when we give the program the next figure (Fig + 2*1) as another 
example of not-ui'ARCU, it again modifies the ARCH ctescriptfon, 
this time i-c quiring (rather than Just mentioning) the supported-by 
relations. 
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Again, this change occurs because the change in support is the most 21 

prominent difference the comparison program found. Finally, we~~ 
show it one more example of an ARCH, and il re-compiles' a de- 
scription in which the requirement that the top- object be a rectangu- 
lar brick has been removed (Fig, 28) , 

There are many arbitrary elements in how this program decides 
what to no at each step — which differences to give highest priori- 
ties, how to match up different description network*, what explana- 
tions or excuses should be assigned to the differences it notices,. 
These commitments arc kept on back-up trees* so that it is possible 
for the program to recover from at least £ome disasters. The goal 
is to obtain behavior that would be plausible in, if not typical of, a 
child. The particular concept that the program develops from a cer- 
tain sequence of examples will depend very much on those examples 
and on the order in which they are presented — as well as on the 
connection of concepts the program has already acquired at that 
time. The experimenter will not always get the results he want3 or 
expects! We cannot expect the first real concept-learning programs 
to be foolproof, any more than a teacher can expect his favorite in- 
structional technique always to work; but here at last we have a 
clianee tp understand precisely how a training sequence interacts 
with built-in and previously acquired ingredients of the system. 
Winston's methods are related to earlier ideas like those in Newell 
(Ref. 3) and emphasize the use of explicit descriptions. While there 
is some similarity* in strategy, to the Feigenbaum- Simon EPAM 
scheme (Rcf. 4) f the result is pointed toward a true structural net- 
work description of the concept. It should therefore lend itself bet- 
ter to direct analyses of examples, Instead of having to work 
through synthesis by generating and testing proposed examples. 

More generally, having a description (rather than just a teat} for a 
concept seems absolutely crucial. The advantages include; 

1) The possibility of making deductions about the concept, 
including its consistency with a proposed detection scheme; 

2) Combining several descriptions in non-trivial ways; 

3} Comparing and contrasting descriptions, as In Evans's 
program; 

4) Using the description to generate, rather than merely to 
select, what next to do in a search program. 

Similarly, in search processes, one could combine several descrip- 
tions to obtain summaries of the information acquired in exploring 

different alternatives. In most earlier heuristic search schemes, 
the procedures usually have to abandon almost all information ac- 
quired in the course oF unsuccessful expiatory attempts because 
of inadequate descriptive facilities. 



22 The A J. Group is now committed to a bruad utturk or. the problems 

of symbolic Learning, through the application of heterological kinds 
of knowledge to the analysis of descriptions. An essay (A.I. Memo 
185) gives a preliminary statement of our plans for this project. 

Theorem-Proving 

Go raid J. Susa-man has implemented a theorem -proving program that 
uses the Resolution principle with an assortment of retrieval meth~ 
od3 and oilier heuristics to mate deductions in predicate calculus. 
Although there has been a uumber of Interesting results, we never- 
theless believe that the use of this technique* as an approach to 
artificial intelligence, is receiving much undue attention today, and 
we do not plan to give it a large place in our future activity unless 
some new and impressive demons Lratipn of its power comes to Light, 
Suss man has been experimenting with a variety of means for com- 
bining the deductive strength of predicate-calculus resolution with 
heuristic flexibility of less formally constrained problem- solving 
schema^ inn his conclusions are not. Mncour/igmg. An >:*aTnple that 
puts one of the problems into a nutshell is this: suppose one is 
g Iven 

A => B and 

C => Dj and 

(A => B) & (C =* D) => E t 

and one wants to deduce the simple c:onclusipn 
E. 

TO be sure, the program manages eventually to obtain E, but only 

after many steps of converting the given statements into expanded 
"normal" forms that are In themselves rather meaningless* 

This would net be *o had in itseU, but it would become an acute 
problem If one were to try to give the theorem -prover additional 
advice about what to dp in other situations, since the kinds of situa- 
tions for which we can describe such advice arc hard to represent 
in terms of the fractured internal expressions the Resolution sys- 
tem creates for its own use* In another experiment Sissman modi- 
fied the representation of this problem so that the => symbols were 
not recognized, by the prover as meaning; implies, and he added a 
separate set of axioms for using a more natural deduction method. 
Now the Resolution system produced a better proof in less time^ 
even though it had to work indirectly through the Jiew axioms! No 
doubt this particular problem could be ameliorated by some variant 
of the many combinatorial schemes that are being widely studied to- 
day,, but we feel that, once such systems attempt to solve "rear 
problems, all such devices will fail as mere atop-ffapts; they do not 



help one to come to grips with the construction of the kinds of cog- 23 

notive models we think are needed to solve hard problems by using 
accumulated knowledge and experience. Sussman's system taJses 
some steps in this direction by maintaining its statements in a 
strut Lure partially ordered by the substitution- instance relation. 
Nor Is his system restricted to first -order predicate calculus* But 
our gloomy expectations remain. Incidentally, we do not feel that 
completeness , or even consistency, is of very large importance. 
Logical completeness is more or less inevitable in any system that 
knows a great deal, and consistency is not a notable feature in the 
intelligent machines that already exist. (The backer* of the Resolu- 
tion method achieve the wrong kind of completeness; the kind of 
comply tEmesu we need ts lor the problem -solver to be able to use 
any "natural* technique.) 

At perhaps another extreme, Carl E, Hewitt is developing a language 
for constructing deductive systems with the utmost heuristic flexi- 
bility. His language, PLANNER, is designed to allow use of knowl- 
edge and types ot representations, uf great variability, PLANNER 
must be one of the least procedural language^: to a large extent, 
one can specify what une wants done rather than how to do it. Con- 
sider, for example, a statement of the form "A implies B\ As it 
stands, it is a simple declarative statement. But In PLANNER it 
can instead be interpreted as the imperative; "set up a procedure 
that will see tf A is ever asserted, and if this happens assert B 
also + * Or, it can be interpreted; "set up a procedure that will see 
if B is ever desired, as a goal, and if so assert A as a new sub- 
goal to be deduced. " This is only the skeleton of the idea: PLANNER 
contains machinery for easy manipulation of many different roles of 
declaratives, imperatives, goals and deductions* In attempting a par- 
ticular deduction, for ejtample, programs can specify suggestions for 
ether theorems that should be used (and even in what order) to 
make the deduction. All statements are expressed in a powerful new 
pattern- matching language, MATCHLESS, in which control of proce- 
dures is specified in terms of the forms of assertions, rather than 
in terms of particular assertions. 

Because problem -solving often requires building up elaborate tempo- 
rary structures, PLANNER has machinery for handling statements 
that were once true in a model which may no longer be true after 
actions have been performed, and for drawing conclusions thai may 
have to be deduced from such a change. Control of such matters 
is specified in terms of local states „ to which are bemud informa- 
tion about changes in the data base -- erasures, assertions, new 
definitions, etc. -- since the data were last updated. 

Hewitt describes his system qualitatively in a confer ltils paper, 
and in detail in A.L Memo 168, He has now implemented 



the language and is programming a compiler to obtain sufficient 
speed to permit full-scale experiment*. At present, he is using it 
to study deduction* about the manipulations of objects by a robot, 
as in "pick «p all pairs of tubes that are the same color, with one 
cube on top of a third cube that is in front of the other," Terry A. 
Winograd is completing a system that will translate such natural 
English statements into PLANNER assertions ail d theorems. 
Natural Language Systems 

Winograd is completing a system tor handling problems of computer 
understanding of natural language. The system is designed to accept 
information in normal English sentences, to answer questions, and 
to execute commands, using semantic information context to under- 
stand pronoun references and to disambiguate grammatically com- 
plicated texts. To do this, Winograd uses a new linguistic scheme, 
based partly on the systematic grammar described in Halliday fi967 
Ilcf. 5) and in Winograd (W, Ref. 6) and partly en a special repre- 
sentation for both the grammar and the semantics- In previous at- 
tacks on such problems, some workers have used heuristic tricks — 
key words or matching of phrase fragments --or thev have used 
non-neuristie formal grammars to do a detailed analysis of the sen- 
tence to which the semantics are to be applied, Winograd's system 
is based on a heuristic grammar that uses contextual information in 
analyzing the sentence, carry ing out the semantfc analysis concurrent- 
ly; this is made possible by representing the grammar as a program 
instead of as a set of static rules. 

Definitions of words, as well as the system's knowledge about things, 
arc also stored as programs and are available to a deductive part 
of the system, This allows much more flexibility than one gets natu- 
rally from networks or rule -lists. The program can make long, 
complex deductions in answering questions or in absorbing new in- 
formation. 

The grammatical part of the system j* operating, with a quite com- 
prehensive English grammar (for a description, see Winograd, "PRO- 
GRAMMAR, A Language for Writing Grammars* 1 , A.I. Memu 181). 
The semantic programs are stiU in preparation, to he combined with 
the deductive system, which will use Hewitt's PLAnnrr language. 
The entire system could be used for a general question-answering 
facility for any corpus of knowledge programmed into the deductive 
system. The first applications will probably be concerned with in- 
structing a robot and with analysis of children's stories at the first- 
grade levels. 

Loosely Stated Mathematical Probl ems 

Several years ago, Dantel G. Bobrow completed a program that was 
able to solve some algebra problems stated in ordinary English. The 



mathematical mate mi was rather sharply restricted to converting 
the sentences into simultaneous linear equations. Recently, Eugene 
Charniak has completed a new program that can solve some prob- 
lems like this, stated In imprecise English: 

A train, starting at 11:00 aoia*, travels east at 45 miles per 

hour while another, starting at noon from the same point, 
travels south at GO miles per hour. How fast arc they sepa- 
rating at 3:00 p.m.? 

Solution ol this problem requires a number of intellectual skills* 

Among these are, of course, Hie ability to manipulate the English text 
to derive a we 11- formulated symbolic problem, and knowledge of ele- 
mentary calculus necessary to solve the symbolic problem* Perhaps 
less obvious , but equally important* is access to miscellaneous knowl- 
edge about the real world; for example, the knowledge that east and 
south are orthogonal directions, and enough knowledge about time to 
deduce that 3:00 p.m< is four hours later than 11:00 a.m. 
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Charniak f & system, CARPS (CAlculu9 Rate Problem Solver) (MAC 
TR-51J, translates this problem into an internal representation of 
which a general impression can be gleaned from Fig. 2S. This struc- 
ture indicates two objects, TRAIN and ANOTHER. Associated with 
each is a velocity given as direction and magnitudej a TIME that 
has a starting value tor each tTain and a WHEN- CONDITION. The 
latter refers to the fact that most calculus "rate problems" ask 
questions roughly of the Jorm + "What is A when B?"; in our case, 
we obtain, "What is the speed at which the trains are separating 
when the time Is 3:00 p.m. t ?" 

CARPS uses this information structure to formulate an algebraic 
manipulation problem* HOW FAST and SEPARATING indicate that 
what is desired is the rate of change of the distance between the 



a* two objects. The distance is the hypotenuse of a right triangle with 

le^s coward EAST and SOUTH. The lengths of the legs are obtained 
by multiplying the rate of travel by the time. The expression for 
the derivative is therefore 

3T V^tt-H].) 2 + (60 [t-T2])2. 

The desired value is at 3:00 p.m. (our WHEN- CONDITION). The 
derivative is found by algebraic manipulation* and the program then 
types 

THE ANSWER IS 72.246 MILSS/HOUK, 

For its algebraic operations, CARPS uses parts of the MATH^AB 

programs described elsewhere in this report. CARPS has been used 
to solve other problems taken verbatim from calculus textbooks. 
These problems deal with cones, spheres and shadows as weU as 
distances. In each case, the program was given sufficient knowledge 
U> ?.ir 3 c Lhi- -<MLk-ji:: = !H, sr-t ■.:,: ,.r. i^L-niil structure, arid ^cn-iitu ' 
the equations. 

We do not wish to imply that the program is very strong at solving 
calculus problems. Frequently, a problem cannot be solved because 
the program is unable to handle the syntax used, no method of solu- 
tion is known to the program, or the problem requires facts about 
the real world which the program does not know or could not handle. 

An example of the last difficulty arises in: 

A ladder 20 feet long lesuis against a house. Find the rale 
at which the top of the ladder is moving downward if its foot 
is 12 feet from the house and moving away at the rate of 
2 feet per second. 

Most adults have little difficulty in visualizing the situation described 
in this problem. CARPS is stuck because it does not know in which 
direction the foot of the ladder is moving. The phrase MOVING AW Al- 
ls interpreted by people to mean "moving away along the horizontal 
ground on which the foot of the ladder is presumed to rest". CARPS 
however, does not realize that. " * 

Clearly, CARPS r s lack of such real-world knowledge cannot be cir- 
cumvented or ignored. A crucial part of research towards problem- 
solving ability of thi& sort is concerned with the representation and 
use of such knowledge. 

We have discussed (A,l. Memo lflS) preliminary studies attempting 
to analyze the sorts of knowledge used by a first-grade child in un- 
derstanding children's stories. Much of the calculus student's "gen- 
eral knowledge' Is already structured at the elementary- school level 
and we feel that progress in this area is essential to the future 



development of anything like "general intelligence " . We have decided 
to make this area a very large part of our work in the next few 
years, 

MATHLAB 

MATHLAB in an interactive computer-program system that is being 
developed to facilitate creative; work thai involves extensive manipu- 
lation of symbolic algebraic expressions. taATHLAB la Intended to 
be of lielp to research mathematicians and to applies of mathemat- 
ics engaged in » heavy manipulative work\ In the MATHLAB project, 
therefore, serious attention has been paid to human engineering and 
to efficiency and speed of operation as well as tu basic mathematical 
problems and to the problem of programming the computer to exer- 
cise initiative and sume judgment in selecting; and carrying out trans- 
formations of algebraic expressions* 

MATH LAB is our best illustration of the idea of building "knowledge" 
into computer programs. The MATHLAE programs are actually quite 
capable in solving certain non-trivial mathematical problems. The 
essential idea* however, is to Include provisions for interaction with 
that knowledge so that the user can build and administer complex 
but well-uuuurytpod procedures. At best, the user and the programs 
supplement and reinforce one another and march rapidly through 
symbolic -manipulation problems that would take the unaided mathe- 
matician countless hours. 

This last year* a new and much advanced MATHLAB was planned 
and partly implemented. Progress was made on faster parsing algo- 
rithms and on a representation of polynomials that speeds up addi- 
tion and multiplication. At the same lime,, a significant advance in 
programmed symbolic integration was achieved by implementing a 
dseieion procedure, due to Riscb, for expressions involving rational 
functions r logarithms and exponentials. 

Also this last year, an extension of MATHLAE to handle special 
functions defined as integrals has enabled Moses to verify or cor- 
rect some published tables of integrals, in 1&5B, 1st, p. Mauxer at 
the Argonne National Laboratory compiled a table of 150 integrals 
involving the crxor function erf(s)* Maurer was unable to verify, by 
hand, the following integral, which he included in the compilation 
with a note to that effect* In fact, it was gliglitly incorrect as printed: 

*2erf{ax + h)e p, *tk = 1 erf (ax + b) E P x (2 - Zp* + p 2 * 2 } 

. pax, - pb + p g /2a - 2a -(ax-ftj^px 
a2 p 2/ f 
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2 a Moses's program was able to find the error. 

Hers is an example of factariitatiun of a polynomial; 
(1) x**e - i - (typed i fl j 

(2} x 6 - 1 = (MATHLAB's response). 

In line 1, the user typed an equation. (The syntax is awkward be- 
cause the typewriter is a one-dimensional device.) Line 2 shows the 
computer's response displayed in the conventional two-dimensional 
syntax ol mathematics. 

(3) 'PF('WS) (input} 

(4) £k-TJ(x+1)(x 2 +)(*1Kk 2 -x+1) (response), 

In line 3, the user asked the system to (actor the equation in line 2 
by requesting the operation PF (Polynomial Factorisation) to be ap- 
plied to tlie WS (Work Space). The Work Space is always the last 
expression known to tha? system. In the response given in line 4, 
the quadratics are not fully factored because the program is re- 
stricted to finding the smallest factors that have integer coefficients. 

Nejct we shall ask the system to inte grate an express ion, differen- 
tiate the result, and then check to see whether or not the result is 
identical with the original expression, 

(5) l/fx**3 + A*x**2 + K } (input) 

(6) ~z ~~i (response) 

(7) 'Integrate ('US, x) (InputJ 
(6) IS THE EXPRESSION 

A* -4 

t& be considered positive, negative or zero (rejpnnse) 

(9) NEGATIVE (Input). 

The computer wants- to avoid terms in the integral which have com- 
plex values, tX possible. T/his is the reason for the question posed in 
line 8, Line & is the user's response, and line 10 i& the integral. 
Obtaining the result involves use of a subsystem for handling partial 
Iraetion decompositions and one tor integration of rational functions: 

(10) - 1/2 log{x 2 + Ak + 1J 

+ ~? art tan £ * ■ -5 — + logfxK 

sqrt(-A £ +4) sqrt(-A ri +4) 

We Shall now differentiate the integral. 



(II) 'DEHIVCNS. *) (Input) 

{1£? - 2A + -l/g(Zx4fl) | 1 
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Well, the result is certainly not identical with our original expres- 
sion (line &). Tills is because the differentiation program differen- 
tiates a sum term -by -term without combining the results. (Note that 
the log * term in line 10 gave rise to the term in line 12.) To our 
rescue comes a simplification program RATSIMP (ftATional Sttlpliti- 
cation) which will expand denominators and combine the entire result 
Into a single fraction. This fraction Is simplified by removing the 
greatest common divisor of the numerator and denominator* 



03) 


'RATS MP ('US 


,«) 




(fnputj 


(14) 


1 
1 ~2 






(response) 


Here 


is another 


Integration 


problem: 




(15) 


X**3/<1 - X 


**g}**(3/2) 




(input) 


(16) 


* 3 






(response) 



0*2) J/Z 
(17.) 'InttgrafttfHS, *) (Input) 

(16i sqrt(~x z + l) + J (response), 

sqrt(-A +1} 

Note the syetenVg preference for -or + l over the more convention- 
al 1 - x K Mathematicians today tend to obey the rule: If a sum of 
two terms contains a positive term and a negative term, write the 
positive term first. Graphic heuristics of this sort are now Included 
in Martinis more sophisticated display routines. 

(19) ( 2*x**6t5*x**4+x** 3+4+X++2+1 )/ lx**Z+l ) **2 *e+*j<**2 

(input) 

— * 5—- e (response) 

{21) 'integrateCHS* x) (input) 

(22) 2k * 2x 3 j 1 k 2 , . 

5 e (respon&fr), 

2*^ + 2 

These results depend on a powerful patter n-maiching program for 
algebraic express 3 ons. 



30 This example points clearly the need (or a. better way to enter 

mathematical expressions into the system* Methods, that allow users 
to hand- write algebraic expressions, using a pen- like device , are 
now ckrae to the step ol useful application (Ref, 7) + 

The fallowing example shows the system solving a linear differential 
equatioi] with yontilant coefficients* a problem-type of great interest 
to electrical engineers. 

(23) DERIVfy,*^) + A*DERrV(y,jO - sfn(2*x} 

(input) 

q 

(24} ^ + *$[ = s1n < 2 *> (response) 

Ojc j u * 

(25) 'LD€SOLVE('HS T y, *) (input) 

(26) NEED INITIAL CONDITIONS (response). 

The program allows one either to enter specific initial values for 
Y(0) d Y r (0), and Y B (D) or to leave them as i determinates (aa it 
done below). 

(27) ALLFORHAL (Input) 
(26) IS THE EXPRESSION 

A 

TO 8E CQNSICEfitD fDSinvt* NEGATIVE OR ZERO 

(reSpqnse). 

Ah before, the answer to this question will be used to generate an 
answer which contains no complex terms. 

(29) POSITIVE (Input) 

(3D) ZY[C) * 2Y"(P) + 1 
2A 

. -Y'(0)A * 4Y"(Q) + 2 , M ,.* , 

+ ^ f 2 — k f cos(sqrt(A}ic) 

A £ - 4A 

+ r«) ^SP- ♦ ^ cm 

(response). 

The solution is obtained with the help of a subsystem that tafces 
Laplanj transforms of both sides and obtains the inverse Laplace 
transform of their ratio, using the package far integrating rational 
(unctions. 



Hierarchical organisations such as MATHLAEVs will become increas- 31 

mgly popular as programs and systems get more complex and so- 
phisticated. While this ay stern is ostensibly working at the higher 
levels oi the hierarchy it is actually spending most of its time at 
the lower support ing level*. Probably, over half the routines of its 
more than 60,000 "words of memory are being utilized in solving a 
differential equation. This should serve as a warning to designers 
of time-H haring systems who would prefer that programs limit their 
use of memory. 

The Held of computer -aided instruction (CAT) has not progressed to 
the extent that many have wished. Teaching programs are unable to 
answer questions other than those that the designer had foreseen -- 
not only because these programs do not know enough English to un- 
derstand the quention, but basically because they do not at aj] under- 
stand the field about which they are- suppled to teaclu The designer 
of the usual kind of CAT course writes a script which is controlled 
by a content -independent interpreter. The script designer must pro- 
vide (pr many possibilities at each step because he cannot rely on 
the interpreter of the script to know anything about the field with 
which the script deals* If a student Is allowed, as he rarely is> a 
reasonable flexibility in replying to the program, then either the 
script de&igner is physically exhausted from considering all the 
plausible replies, or he misses considering many such replies* We 
contend that a primary step in writing a sophisticated teaching pro- 
gram should: be the education of the program in the area about which 
it must teach. 

Consider,, for example,, the problem of writing a program far the 
teaching of freshman calculus techniques far differentiation and inte- 
gration. Using the- knowledge of these techniques that is embedded 
In the MATHLAB system. It would be possible to write teaching 
programs that check the steps of a students calculation, advising 
him of errors as he progressed. A preliminary program with these 
facilities has been prepared, Such a program could also be capable 
of performing differentiation or Integration problems suggested by 
the students ^ explaining its steps as iL proceeded. To achieve such 
capabilities through the writing of mindless scripts is unthinkable. 
TO be sure, the experiments with the Calculus Hate Problem Solver 
(CARPS) have shown that a great deal more must be known before 
such programs can deal effectively with human beings in a wide 
area of knowledge* 

The task of finding out what human users know is clearly related 
to the tasks of workers in the field of. Unguis tics, psychology* and 
philosophy. A major tailing, however, in the education given in these 
traditional disciplines is the lack oi understanding of the concept of 



32 an algorithm or process. We believe that advances in our under- 

standing of human knowledge as algorithmic processes are likely to 
be of supreme importance., 

Although the moat essential part of MATHLAB is conceptual and the 
next most essential is the programming of symbolic transformations, 
the interface with the uaer is very important, and we Look forward 
to giving MATHLAB every advantage of effective man- computer- 
interaction tectiniqu.es. 

Chess 

MACHAC-VI, a chess program developed by Richard D. Greenfalati 
lor tlia PDF- 6 computer h was btgun in November 1966. It entered 

its first tournament the following January. Since then, it has par- 
ticipated in lour more official U.S. Chess Federation tournaments' 
and has achieved an official rating ol 1528. In its last tournament, 
it had! a performance rating of 1720 and drew an 18&0 player, These 
statistics put it a little more than one standard deviation below the 
mean strength of all U.S. tournament players (which is about lB0Q} x 
The machine has played 'perhaps 2000 games against chess players 
of every strength, It wins about 86 per cunt of its games against 
tournament players, Greenblatt estimates that it represents a pro- 
gramming effort of about six man-months. 

Once the basic program was completed, the llrst step in refining 
its play was to give it a model ol the flow of power on the chess 
board. Next, it was given explicit techniques tor pins and discov- 
eries; then ft was given positional and pawn structure considera- 
tions. These and other features are largely completed now. 

The next major step, only partially Implemented now, will bo to in- 
ter par ate symbolic reasoning ability into the program. The oppo- 
nent's threats are specifically identified, and each reply is evaluated 
for its ability to answer some or all of the threats. This does not 
mean that the opponent's threats were ignored before; but now the 
program proceeds with a more explicit sense of what it is doing, 
whereas in the claHsical mint max strategy- the threats are implicit 
and therefore cannot be subject to symbolic statements ol what to 
do. The explicit system should be much more efficient. 

Another new feature, already implemented, adapts the plausible- 
move system to recognised states of the game: if one side appears 
to be ahead in the look-ahead analysis, it will prefer holding, trad- 
ing and simplj fying moves; the other side will rehire positive, at- 
tacking moves that might yield tangible gain. These and other fea- 
tures are operational, but the other programs have not been modi- 
fied to take as much advantage of them as they probably could. 



Another class of improvements is planned which will include real 33 

symbolic learning rather than mere tuning-up. We expect that these, 
with improvements in end-game strategy, will result in a substantial 
increase in strength of play r 

Eye-Tracking 

The much-used term " man- machine interaction" usually refers to 
situations in which the communication relationship is exceedingly 
lopsided,. Man can see, hear and touch the computer in many ways 
and places j. hut the machine is restricted to receiving information 
through a narrow bottleneck — usually a Teletype, We have been 
interested (or a long time in redressing this imbalance. This year 
we were able to achieve an old goal of enabling the machine to look 
at a person. More precisely; the machine look* at the man's eyes" 
to determine his point of fixation* 

Recording eye movements is a well-established technique in the 
study of perception, tracking skills, and even problem-solving be- 
havior. But traditional methods give the pattern of eye movement 
only alter analysis. We believe our system, which Incorporates an 
eye -tracking device developed by J. Merchant of Honeywell under 
NASA contract, j^ the first that enables a computer to use the In- 
formation in real time. As an example of its uses r Samuel L. Gcff- 
ner has two programs that display text for a subject (such an a 
small child) to read and take action related to the word currently 
fixated. One of the programs causea the word to be pronounced by 
the computer; the other causes the word to be replaced, in the dis- 
play, by its translation into another language. These programs are 
mere demonstrations --but they illustrate rich applications to 
teaching and other interactive situations. We are pursuing such ap- 
plications, partly with support from NASA. 

The A J. Time -Sharing System 

The experimental work of the Artificial Intelligence Group requires 
a hi^h level of computer service for a limited number of users. 
Our system is based on time-sharing a two-processor machine 
{FDP-6 and FDP-10) with a core memory of 2 l ® words of 36 bits. 
WOrmally, the programs of the active users remain in fast memory; 
this limits the numher of users now, but a paging device to be com- 
pleted early in 1910 should allow some expansion required by the 
maturation of certain projects, notably MATHLAB. 

The special requirements of the project led to a time-sharing sys- 
tem with enough novel features to merit discussion. Donald E, East- 
lafce describes the system in detail ("ITS 1,5 Reference Mannar, 
A T L Memo 161a) t Besides the usual kinds of input-output and system 
calls, theTe are special calls to reduce overhead and to facilitate 



34 real-time control. These include programs tor operating the mechan- 

ical hands and computer eyes and other special remote - control de- 
vices, AU ordinary time -shared programs run in one of the proces- 
sors* and critical real-time processes are assigned, by use* calls 
to the other H The real-time processor is normally assigned to a 
single user at a time. 

Because all user programs ordinarily reside in core, it is possible 

to switch between programs with great rapidity. Quanta of user 
time are short enough to allow program response to single typed 
characters without noticeable delays, When swapping is introduced, 
we expect it to affect only semi-dormant users. 

A user may have many jobs running " simultaneously", Each user 
commands a tree of procedures; each can create and control sub- 
ordinate procedures; all have equal access lo external devices and 
files. The top procedure, loaded automat ieally when the user de- 
clares his existence, contains a version of the well-known DDT de- 
bugging system. A special character allows transfer to the top pro- 
cedure from any level, so that the user is automatically in position 
to u&e DDT's interrogation, breakpoint* an d Qt her debugging features. 

Input-output devices are referenced symbolically and data can be 
transferred on character t word or block bases; an entire video 
image can be acquired by a system call while the calling procedure 
continues without waiting. User programs need no buffers in theSr 
owiti core images* Line-printer requests, for example, are buffered 
until the device is tree* User programs communicate by "software 
interrupts* that are treated the same as hardware interrupts; superi- 
or procedures can store and retrieve words in interior procedures 
as though they were I/O devices; buffered communication is pro- 
vided so that pairs of procedures can treat each other as input de- 
vices. The file system resembles the CTSS file system. 

The schedule algorithm tries first to equalize time between users, 
and second to equalise times between the procedures of a single 
user tree. A special procedure always runs which performs various 
jobs and can check constant portions of the system against a copy 
in an attempt to detect some forms of hardware or system failure. 

The secondary storage uses IBM 3311 discs and DEC mierotape 
drives whleh are file-structured in the same way, Users can estab- 
lish symbolic links between file directories so that files can be 
shared by many programs and many users; these links can he 
chained. 

, The system lias several operating stations. These include four text- 

display devices, several Teletypes and external telephojie lines, a 
main console with DEC 340 display and several slave monitors 



around the laboratory > and a special console with controls for oper- 3S 

ating the eye -hand system. The system interfaces with a. radio trans- 
mitter lor inexpensive long-distance remote experiments* 

A multiplexed digital -analog system operates either in word or blocJt 
mode, on call or automatically, as requested, Any number of users 
may simultaneously have analog access on different channels in dif- 
ferent modes. These. are some of the real-time facilities: 

1) Iris, focus, stereo mirror;, for high-resolution image dis- 
sector, 

2) Pan, tilt, zoom, focus* Iris, for small, low -resolution 

image dissector, 

3) Extend T tilt, rotate, grasp> finger curl, lor hand MA- 3, 
4} Extend, tilt> rotate, grasp, for hand MA-2, 

5) Roll, yaw, horizontal, vertical, swing, lor arm MA- 3, 

6.) Position for tactile -sens or device, 

7) X, Y, Z, rotate for arm MA-3, 

For moving the arms, special system calls provide acceleration- 
and velocity-limiting, software limit stops and other performance 
limits. To prevent disaster, certain motion commands are uninter- 
ruptablo for limited times, and an arm-moving call is illegal if the 
device is utider control of another user. Motion calls are not auto- 
matically buffered like those to the line printer! Although one can 
read any input channel at any time, most mechanical devices can 
be operated at leisurely speeds, with the Input multiplexer channels 
read automatically. The system normally does this at a minimum 
of 5 times per second. 

On the input side are sensors for all mechanical de-gre.es of free- 
dom, including hands and arms and: optical parameters. TheTe is a 
test stand for calibrating the positional control of the arms, and 
there is a variety of portable remote -control boxes with switches 
and continuous-adjustment knobs. A special system call gives the 
user great flexibility in real-time control of program parameters. 
It connects potentiometers {through the input multiplexer) to arbi- 
trary program variables -- i.e., memory registers -- and these 
can be specified to be fixed or floating, or arbitrary bytes. There 
are options for assigning absolute or incremental meanings to knoh 
positions; in the incremental mode, there is a programmed velocity- 
dependent gain and a side -to-side hysteresis so thai it is easy to 
make fine and coarse adjustments with the same control, and it 
tends to remain centered in its motion range. 



3* Tht fey stem contains good facilities for graphic display, which are 

presently limited to one user. Growing needs dictate acquisition or 
multiple- display Hardware. 

LISP (MACLISP) 

UBP is the high-level language the Artificial Intelligence Group 
uses. Our LISP version is different in many ways Irom other 
LISPs, und some of these differences represent progress, The de- 
tails are of interest only to specialists, who may consult the memo 
by Jon L. White { fl Time -Sharing LISP for the PDM", A J. Memo 
190) ♦ The system does not use an A- list for ordinary variable bind- 
ings; It uses a value property; a special stack is used to unbind 
after lambda- conversions* There arc many small improvements in 
human engineering such as default valuer of non- spec Hied function 
arguments. The garbage collector manages apace for arrays. The 
HEAD program allows new character- handling facilities and macro 
definitions of certain kinds. There is a variety of powerful editing 
facilities and display packages, and some strong debugging systems. 
The machine- language feature „ LA1% has been strengthened. Whit- 
field Duffle has restructured the compiler to mate the most of its 
decisions by dispatching on tables so the users can introduce special 
forms by giving the compiler instructions through the table entries. 
Ic would be more efficient tf the compiler were able to make a 
deeper and more systematic analysis of the uses of expressions be- 
ing compiled, separating executions for value from those for effect 
On flow of control, for side effects, etc,, and for recognition of un- 
necessary recursion and similar simplifications. This itself is a 
problem in artificial intelligence. 

Motivated by MATHLAB and other projects, we are now making an 
effort to introduce facilities for very fast arithmetic into LISP, We 
expect that the resulting system will be almost as efficient at num- 
ber-crunching computation as is any conventional algorithmic lan- 
guage, 

Mechanical Hands and Arms 

The project has developed several mechanical effector devices for 
computer-controlled manipulation, This field is still substantially 
unexplored, and our work can be considered only to clarify some 
of the problems , not to solve many* There are problems fn several 

different areas. 

There Is a need for much deeper analysis of what is needed in de- 
signing a hand-arm system for various kinds of jobs. In his thesis 
David L. Waltz studies motions the human hand uses to perform 
several basic loul-to-udling and similar tasks, to see which of the 
degrees of freedom of the hand were most essential, and how they 
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pedic literature, but ii is not sufficiently structured to serve as a 

base for programs. One needs to describe actions 1 ti terms of inter- 
actions of position force, velocity and sensory responses -- in short* 
cne needs something like a programming language foT It. We also 
examined some choreographic languages but, although they contain 
some good ideas about path -of- motion description, they do not have 
sensory- interaction predicates. Waltz developed some notation that 
seems helpful and Ernst's (Ref, fl) discussion is still relevant, 
Waltz's (earlier) thesis yhows how a lew degrees of, freedom can 
accomplish much, but out wishes there were much more known about 
this subject. 

It: il:u^I :if :uir i:-au..t insula] work, we Illlvn ^nvnkiyed a modified 
AMF Versatran industrial manipulator arm (which operates in cylin- 
drical-polar coordinates). Although the hands we attached to this 
device have some extra motions, the larpe -scale motion of that type 
of arm has no such redundancy ; thus there is basically only one 
way to reach each point in apace. Calculating this is straightforward, 
solving a not-too-complicated equation that has only one solution. 

Because this Is a very clumsy device -- such an arm is unable to 
reach around obstacles or even to support an unobstructed object 
Irom an arbitrary direction --we developed a much more mobile 
arm with many extra degrees of freedom. The new arm behaves 
more like a tentacle than like a coordinate system, Hydraulically 
operated, it is relatively slim and has five articulations, each with 
two degrees of freedom, a shoulder, three elbows and a wrist H An 
articulated prototype shows dearly that this geometry is adequate 
iar everything a human arm can do and more. In toe actual hard- 
ware mechanism, each of the eight interior hinge joints has approxi- 
mately 110 degrees of flexion. We conclude that this is inadequate; 
it must be more nearly 1G0, The lesson we learned is that in a 
minimal system a small angular restriction means only that the 
work apace is somewhat reduced. But, in a linkage whose purpose 
is a great variety ol ways to reach poinls in the work space, all 
restrictions interact in messy ways to break up the higher -dimen- 
sional mobility space into hard-to -understand fragments, so that two 
slightly different Ihrcu -dimensional positions may have to be reached 
(if at all) by grossly different global arm configurations. For such 
a mechanism with 10 degrees of freedom for reaching points in only 
three dimensions, there is a mathematical problem of inverting the 
position equations, One can sometimes get by with simple hill- 
climbing by successive approximations, but thiR is really unsatis- 
factory because it is hard to adjoin global information, and it does 
not give an analytic picture of the alternative solutions to the prob- 
lem, Jean- Yves Gresser, in his (earlier) thesis discusses a method 



3B that gives a solution specific to Our arm- mechanism, using a rather 

general method of dividing the position problem into a series of 
almost -independent factor sub-problems. Wliat Greater does is to 
divide the arm in hall, and describe the mobility space of each hall, 
using simplifying approximations for each aide. When the haif- 
solutions are put together, there needs to be an accuracy- increasing 
iteration anyway, so there is hardly any real loss through approxi- 
mating. 

Now, in our arm- mechanism, it happens that the- mobility zones of 
the half-amis can be described In term* understandable to humans -- 
for this particular arm, each zone resembles a portion of a torus. 
To find the ways to get the whole arm to a certain space position, 
One describes the intersection of the two tori, one -centered at the 
s]ioulder position and the other the goal position. Each intersection 
point yields an approximate solution to the problem (that is, a pos- 
sible location of the center of the arm), and a hi ftfier- level program 
could be asked which solution Is moat desirable. 

There is a larger problem here: how should one represent a ma- 
chine's body image? For the problem of a single, not-too-complicated 

arm, one can doubtless gCL by with cleverly coded, sparse* three- 
dimensional arrays, but one would like something more symbolic. 
And one wonders what happens in the nervous system; we have not 
seen anything that might be considered to be a serious theory. Con- 
sider that a normal human can place an object cm a table, turn 
about and make a gross change in his position and posture, and then 
reach out and grasp within one or two inches ol the object, all with 
his eyes closed! It seems unlikely that his cerebellum could per- 
form the appropriate vector calculations to do this; it is not a mere 
matriT inversion (and, even if it were, one would still need a theory)* 
We would presume that this complex motor activity Is made up f 
somehow, of a large library ol stereotypical programs, with some 
heuristic interpolation scheme that fits the required action to some 
collection of reasonably similar stored act tons + But we have found 
nowhere any serious proposal about neurological mechanisms for 
this, and one can only hope tliat some plausible Ideas will come out 
of robotics, research itself. Unfortunately, at present this area is 
somewhat dormant. 

Another conclusion from our experiments is chat delicate manipula- 
tions must be controlled by application of controlled forces (rather 
than by direct position control), with attention to matching velocities 
with inertias. For measuring the forces on a mechanical hand, we 
have taken two approaches. First, it is quite feasible to engineer a 
grasping surface with good pressure sensitivity at a great many 
points by wrapping a coaxial cable connected to a Time -Domain Re- 
flectometer. {A TDR is a sort of radar system designed to measure 



reflected radio waves that arc produced in a soft- sheathed tube ± by 
any deformations of the wall. The instrument permits hundreds of 
thousands. Df points , using a. single electrical connection to tin: hand. 
(Waltz describes this system in his thesis.) Second, we have devel- 
oped a strain-gauge telemetered wrigt that provides force informa- 
tion in the necessary six degrees uf freedom. This little instrument 
is able to tell where and in what direction the hand Ls being pushed, 
assuming that the force is beiiig applied at a single point ~ a con- 
dition that usually holds in a first contact with an object. 

Solution of the appropriate moment equations for the set of six 

forces at the wrist (fig. 3-0} yields a certain line in three-dimen- 
sionaf space, alone; which a force of a certain magnitude is applied. 
If the machine knows, the geometry of. its hand, it can presume that 
the force is applied whom this line intersects the hand. 
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Fig. SO 



We have built this hand and are testing it. 
a memo by Callahan, Shah and Minsky, 

Computer Eyes 



We shall describe It in 



A computer must have eyes if it is to aee. When the project start- 
ed, no device was available for interfacing a qamputer with a rual- 
time camera, although there were film -reading devices on the mar- 
ket. We first used a Vidicon television camera, but we had diffi- 
culties with the limited signal -to- noise quality df the Vidicon. 
Besides, Vidians are not suitably adaptable to random -access 



40 scanning, they h«ve motion persistence and other problems. We 

decided that this added up to too many obstacles. The image - 
dissector camera offered much cleaner images (at the expense of 
sensitivity, which, for laboratory purposes, was unimportant} so, in 
collaboration with Information International, Inc., we designed and 
constructed our present camera system. An image dissector is es- 
sentially an inverted cathode-ray tube: an image projected on a 
photo-cathode produces electrons which are focused through a de- 
flection system so that the current from a selected image point 
falls through a small hole and is then measured. 

In such a system, the signal-to- noise ratio depends essentially on 
the number of electrons, and one can obtain more precision at the 

price of longer time -exposures t We decided to put this signal- to- 
nolse ratio under direct program control and to make it independent 
of the brightness of the point being measured. This is built into 
the camera's video processor., Horn gives the details ("The Image 
Dissector 'Eyes 1 », A.I. Memo 176). If the constant signal- lo-noise 
constraint were enforced even on very dim points, the ejtposuro 
times would become excessive, so the system has also a dart cut- 
oil parameter that terminates the measurement very early if an 
initial estimate of the brightness falls below a specified value. This 
is also built in the video hardware. The resulting system can meas- 
ure intensity over a 4096; i dynamic range, with a brightness- 
di&criminatlon sensitivity of one part in 64 throughout this range. 
The output appears as a floating point number or as a logarithm 
of intensity. The eye has also a reference -brightness input that 
can be used to make the measurements almost Independent of over- 
all illumination fluctuations, and there are various protective de- 
vices to prevent damage to the image tube by excessively bright 
light sources, Horn's report discusses many advantages and pitfalls 
In using this system; anyone contemplating using image dissectors 
for computer vision should correspond with us about the results of 
some modifications now under construction. 

It is frequently suggested that one ought to build various forms of 
parallel- processing artificial retinas, perhaps along the lines of 
biological systems. But the operations of the vertebrate retina and 
the subsequent image processing is not nearly so well understood 
as is generally believed „ and we do not think the time Ls quite ripe 
for taking such a step in hardware. In our present operations, the 
computation time consumed in the higher levels of scene -analysis 
is comparable to that spent in the lower-ley e I pre-processing, so at 
present there would be no large time factor to be gained from fast 
parallel hardware — even if we were able to decide how it should 
work. 
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Research on Intelli g ent Automata - Marvin Min^ky and Seymour Paper* 

The largest sector ol research In our group is a till the study of methods 
for providing machines with greater visual and manipulative taxabilities. 
Our general approach to this goal was described in last year's MAC 
Progress Report. We expect, some time In 1969 T to demonstrate some' 
practical capabilities of automatic, visually guided manipulation, by showing 
the computer a structure made of children's blocks, and having it build a 
functional duplicate. Details of this work are described separately in the 
Status Reports of the intelligent Automata Project, and iti many of the 
Artificial Intelligence group memoranda available Hi rough otir office. 

Comp utational Geometr y - Marvin Minsky and Seymour Papert 

Preoccupation with theoretical aspects of machine vision has led us to 
crystallise a general concent we tall 'computational geometry". This is a 
new mathematical specialty concerned with the complexity of computations 
necessary to recognize various properties of geometric objects. The rapfd 
evolution of discoveries in this area has given us hope that it will lead to 
new directions for "computer science* in general, by providing subject 
matter that combines intuitive clarity and practical importance, with close 
connections between classical parts ol mathematics - geometry and algebra. 
We helteve the widely recognized conceptual fragility ol current T theories" 
about computation is due in large part to their attempt to build upon ex- 
cessively abstract principles ol automata theory and linguistic structure, 
without enough concern for thorough understanding of particular problem 
areas in relation to particular machine structures. The following sections 
show some examples;. 

A, The SEE Program 

Computational requirements lead to geometric questions of an entirely new 
sort, such as surely never occurred to Euclid. Consider the need to analyze 
a scene in which some of the objects partially obscure other-h.. 




The methods described in last year's report presupposed a computer model 
or description of each kind uf object - cube, pyramid, etc. Since then it 
has become clear, in further work of Adolf o Guzman, that this information 
is not wholly necessary. Tndeed, computations are u&ually much simpler if 
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baaed on a more abstract and general theory of the appearances of objects 
Earlier methods were based on partial recognition ol the bodies such 
the scene below s 



as 




Here, ivhcru 411 objects are rectangular solids, and do ngt occlude one 
another badly, we can discover the objects by the extremely local process 
of locating all the *y- joints,* Each object contains *t most one such dis- 
tinctive feature. This could, of course, fail because of perspective as in 



which could be a cube, or in 



D 



e 



(for we rehire each of the three angles ol a Y- joint to span less than 
lfiO degrees). A more serious failure i& in the case ot occlusion, as in 




where one of the V- joints is completely hidden from view. But the great 
power of programs capable ol hierarchical decisions is illustrated by the 
possibility of first j^ecoffni£k^jhe_small_ J cube ) then rem win e, it. then 
extending the hidden lines, and so discovering the large cube! 





The program developed by Adolfo Guzmin proceeds in a rather different 
way; his idea is to treat different kinds of local configurations as providing 
different degrees, of evidence for fl linking* the faces that meet there. For 
example, in the following three types of vertex cunfijra rations 
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ARROW 



T 



th* *Y* provides evidence for linking I to l[ p II to in, and I to in. The 

"Arrow* just linkg I to II, Because a "T" ts ordinarily the result of one 
object occluding part of another, it is not regarded as evidence linking I to 
III, or n to in (and it is also neutral abgut r and IT), Using just these 
rules, we can convert pictures Into associated groups of faces as follows; 
we represent Y- links by straight lines and arrow links by curves. 





(1) 






(2] 




(3) 





(4) 



So far, there has been no difficulty in associating sets of linked faces with 
objects. The usefulness of the variety of kinds of evidence shows up only 
In more complicated cases. In the Example (E) 
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1^ ^ . ! * Ch * c °"n"tions, the program uses a hierarchical 

scheme that f.rst fines £ ub 5ets of faces that are v*ry u^tlv ifi^ /„ 

linked aces There , S no common in Examples fl)-(4) f but in M the 

™ ,V?h' 7!? 3lmpe nc ° m P etiiim ' algorithm were not adequate here 

one couic a *> take mto account the native evidence the two S 
provide ag ainst linking i-m and rr-m. J 





(6) 



other kinds of Unto, mcludmg evidence from eollinear T-joi.it lines of the 




_ ^ o- — ■ — ® 

® © 

and the elfficte ol some vertices are modified by their associations with 
o^rs This variety of resources enables the program io solve conX 
scenes like that illustrated at the beginning of this action, 

»« *tT WiU J bB in GUKm4nrs doctor41 thesl * ^ appear in the Spring of 

1969. The surpria™* po^^r B nd elegance of this algorithm suggests that 

evidence or h ls come, from the discovery of simple Wlsti,* that seem 
pLfseenet ^ ^^ abDut mifi ^ "«» i* Pi*™ ol com 

B. The Theory of Perceptrons 

^l\T eT f yC ^' WC haV ^ b<?en * nttre * ted in fi^L** a theoretical basis 

for asking ability and limitations of the pcrceptron and similar ma- 
chines These are highly parallel computation schemes that attempt to 
reeogniz* completed inputs by l) computing many properties of the Input 

!l t r\ 'TT^ ^ t0 reco ^^ ** ^en 2) basic* a decision on sLe 
relative simple cognation of the re3ults of lJlfi flrst 
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Our interest tn such machines is not based on very much concern for their 
practical possibilities, for these are very limited. However, it is our con- 
viction that these machines are nonetheless of critical importance a* theo- 
retical models, because if we cannot thoroughly understand yuch stmpie 
computers, we tan have little hope of obtaining pood theories of more . 
powerful and general computers. (The "theory of computahility- for * uni- 
versal* machines is totally unsatisfactory in the context of any real prac- 
tical problems.) Fortunately, we nave obtained a wide variety of theoretical 
conclusions about parceptrons, and these are given tn detail in a new book 
[Perceptrons: An Intro duction to Computational Geometry . Marvin Minsky 
and Seymour Papert, M.l.T. Press, lflflfl). 

We shall summarize some of the results here. First, let us define a 
perce ptron of order K : 

Let R be a part of a two-dimensional plant. Let X be an arbitrary black- 
and-white * picture", t.e H , a subset of R. Any point in X is considered to 
be black,, and the rest of R white. Let $ be a set {$, tp\ $" t . m I f 
predicates - functions whose values are or 1 - each of which depends 
on no more than K points. Finally, choose for each £ a number a. and 
define * 

$'{X) = 1 il £c^(X) < o 

- if Jcr^(X) < 
This -definition includes the concept of "threshold", as in 

Ea^fX} < $ (6" any real number) 
if we allow one of the $ functions to be a constant. 

Now we ask whether a perceptron can recognize a "pattern*. For example 

is there a perceptron such that ^(X} 1 when X is a square (or convex, 
or connected) and ^(X) = when X Is not a square (or not convex, or dis- 
connected)? Our analytic methods depend mainly on replacing the geometric 
concept of a pattern by the algebraic properties of the transformations that 
preserve the features that concern us. We cannot give a full example of 
how this is done tor t;ey metric concepts, but the following sketch shows 
how the algebraic theory works in a fairly simple case. Chapter references 
point to corresponding sections in the book Pereeptrons. 

Theorem 3,1 (Chapter 3) Informal Version 

Suppose the retina R has a finite number of pointe. Then there is no per- 
ceptron £ctyf(X) > d that can decide whether or not "the number of points 
in X is odd" unless at least one of the ^'s depends on all the points or R, 

Thus no bound can be placed on the orders of perceptrons that compute 
this predicate for arbitrarily large retinas. To realise it, a perceptron 
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must start with at least one 4, that looks at the whole picture] The proof 
uses several steps r 

Step 1; In il.i-ii,4, we define 'percepiron\ " order \ etc, more precise- 
ly, and show that certain details of the definitions can be changed without 
serious effects. 

Step 2: In §l + 3 we define the particularly simple 9 lunetions called "masks' 
For each subset A of the retina, define the mask r£JX) to have value 1 if 
the figure X contains or "covers* all of A, value Otherwise. Then we 
prove the simple but important theorem (§1.5) that if a predicate has 
order > K (see SI. 3) in any set OF 4, [unctions, then there is an equivalent 
perception that u^es oiily masks of si&e > K (see S0 + 2) 4 

Step 3: To get at the parity - the 'odd-even* property -. we ask: What re- 
arrangements of thejnput_s pace R leaves It una ffected? That Is, we ask 
about the group of transformations of the figure that have no effect on the 
property. This mi B ht seem to be an exotic way to approach the problem, 
but since it seems necessary for the more difficult problems we attach 
later, It in E00 d first to get used to it in this simple situation. In this 
ease, the group is the whole permutation group on K - the set of all re- 
arrangements of its points. — 

Step 4; In 52 we show how to use this group to reduce the perception to a 
simple form. Tjje_jr r pp p _ fctlvarl;uiC e tni>ore iL1 prQvtfd in fi3j2 is . „ sed to B||0W 
that, for the parity perception, aU_jnasks_with the same sup port size - that 
is J _al(J-hQse that look at the.,samg_ngmbflj_gf_ po liits - ca njbe_i?iysn identical 
coefficien ts. Let 0, be the weight assigned to all masks that have sunwirt 
size = j. 




Group- invariant coefficients for |R| = 3 parity predicate. 
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Step 5; It is then Shown (in 53.1) that the polity pu re eptrun can be written 
In the farm 



p.0?) 



> o, 



where |x| is the number of paints in X, k is the largest support &ize t and 
f'?n IS th9 number Of subsets Of X that have j elements. 

Step C: Because 



(*) - J *(n)( n -l)...U-! +1 ) 



is- a product of j linear termi, it is a polynomial of degree j in n. There- 
fare we can write our predicate in the farm 

P k (1X|) > 0, 

where P^ * s a polynomial in Ixl of algebraic degree < k H Now if Ixl is an 
odd numberj P^(|X|) > 0, while If |Xj is even, P^(iXi) < 0, Therefore,, Lit 
the range S. |Xj < |Rl, Pjj must change its direction |R| - 1 times. But 
a polynomial must have degree > |H| to dp that, &o we conclude that 
k >: \R\* This completes the proof. 




This shows how the algebra works into our theory'' For some of the more 
difficult thflcirHmH we need, somewhat more algebra and gTOup theory. 

Here are some of the positive results: that certain patterns have certain 
orders, 

Patterns ol Order 1 

§P + S "The center of gravity lies to the left of a certain given point on the 
X-axis." 

§2.4 Other similarly defined properties of moments „ in fixed coordinate 
systems. Includes "The area of the image is less than A." 

51.5 Linear three-hofcd inequalities,. 
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Si. 4 -The linage is exactl y a certain one", Qr "differs tram it by not 

more than a given area A." 

P atterns of Order 2 

57.3 "The figure is symmetrical about a fixed point in the plane >■ 

s 7. 9 -Two figures, on two given lines, are congruent under translation/ 
(The coefficients diverge, however, as the retina size grows, J 

51.6 "The area of the image lies in a certain range," 

S6.2 "The figure Ues within an a* Es -parallel line." 

i0.8 The moment of inertia of the figure exceeds some threshold.' 

Patterns of Order 3 

§7.10 "Two figures on two given lines are translation- equivalent with 
bounded coefficients w,. 11 

§6.3 "The image is a convex figure." 

S6.3 "The image is an axis-parallel rectangle/ 

§7*7 "The image Is a square/ (Coefficients diverge ) 

66,3 Any two instant es of figures not translation -equivalent can be dis- 
tinguished. 

Patterns of Orde r 4 

7,7 "The image is a square parallel to the axis." (Bounded coefficients.) 
§6.4 'The image is a circJe." 

§7.5 "The image has a vertical axis of symmetry." {coefficient* diverge) 
(Believed to be unrecognizable in any order for bounded coefficients) 

17.4 Reflection symmetry on a iine. 

55.8 "The Euler Number - that is, the number oi Components minus the 
number of Holes - exceeds a given number. 

Patterns of Or de r 5 (or p ossibly one leas) 

i<:.:> "There is a certain number of convex objects." (No holes permitted) 

13.8 "The total curvature of all the boundaries lies in a certain Interval/ 

57.5 'Two plane figures are equivalent under trans Jation/ (Unbounded 
coefficients) 

Patterns of Order 6 fprrihahR-j 

37,3 "Two plane figures are equivalent under translation inn dilation," 
The remarkable thing about the above results is that the order remains 
fi*ed, regardless of the size al the retina. But for other patterns, the order 
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Increases without hound as the retina is made larger, and it is fair to 
say that these are, In a practical sense, outside the conceptual ability of 
perception- like machines. These Include: 

Patterns of Unbounded Order 

§3.1 "The number uf occupied retinal cells is odd. 1 " (The order is known 
to be equal to the number of points in the retina.) 

15 *The image is a single connected whole** (The order is known to 
grow faster than -/F, probably grows as yN.) 

sS.tf All topological properties except simple functions of the Euler Number, 
For example; "one part of the image lies within a hole inside another 
part* cannot be recognized. 

H.Q Certain simple Boolean combinations of pairs of first -order patterns. 
This and §3.1 show strikingly how perceptions differ from serial 
computers* for these patter an are vury easy for serial machines. 

§6.6 Very few of the finite-order properties mentioned above remain finite 
order in the context of other figures, or Of noise, 

CU Connectedness and Serial Computation 

Ch±r deepest results in the perceptron theory are concerned with the geo- 
metric - or, rather, topological - properties of connectednes s. Because we 
felt that there way some icdierently serial - or recursive - character to 
connectedness, w e decided to investigate the computational geometry of tils 
concept in the context of other mixtures of serial and parallel machine 
structures, Including Turing Machines and Iterative Arrays. Some of these 
are discussed in Chapter 9 of Perocptrons . We can give here an example 
of one such remit, 

Terry Beyer has investigated the time necessary lo compute ^ C(l ^ K iti in * 
situation that provider a different and perliaps more natural model for 
parallel geometric procedures. Suppose that each square of a retina con- 
tains an automaton able to communicate only with its four neighbors. It can 
also tell the state (blacfe or white) of its square h The final decision about 
whether the figure is connected or not is to be made by some fisted automa- 
ton, say the one in the top left-hand corner. On the assumption that the 
states change only at fixed intervals of time, we ask how many time units 
must pass before the decision can be made. It is obvious that on an n x n 
retina this will take at least 2n time units, for this is the time required 
for any Information to pass from the bottom right corner to the top ]eft r 
It is not difficult to design arrays of automata that will make the decision 
in the order of N time units* where N is the area of the retina.. Beyer's 
remarkable result is that (Z + ^J-VN is sufficient, whurc f can be made as 
small, as one likes by allowing the automaton to have sufficiently many 
states. 
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Thus the order of magnitude ol Lime taken by the array is proportional to 
VIN| N which is intermediate between Hie times taken by Hie single serial 
machine (N) and the most gehera] type of parallel computer which is known 
to take < (log NJ t ( Percept rons , Chapter 9) 

The tallowing gives an intuitive picture of Beyers algorithmic process The 
over-Ml effect is that of enclosing a component in a triangle as shown" be- 
low, and slowly sweeping it into the northwest corner by moving the hypote- 
nuse inward. 
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Each component is compressed to one isolated point before vanishing. When- 
ever this event takes place, it can be recognized locally and the information 
transmitted through the network lu the corner. Thus the connectedness de- 
cision is made positively or negatively, depending on whethex such an event 
happens once or more than once. Mo™ precisely, the compression process 
starts by finding the set of aJl "southeast corners' of the figui 



ire T 

hi this theory the retinal points are taken to be a square 
checkL'rtxjard-likc array. Two black squares are con- 
nected if they share a common edge, or are in a chain 
of s glares so connected* (Diagonal corner -contact does 
not count as a connection,) 

The center square is a southeast comer if the South and 
East are empty. All other squares shown may he empty 
or full. 

In each step of the compression operation, every southeast corner is re- 
moved, and a new square la inserted when necessary to preserve connected- 
ness, as shown below: 
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The diagonal lines show how repetition of this local process does squeeze 
the figure to the northwest. 

Repeated applications of this operation eventually reduce each component to 
a single point. The next figure shows how it (narrowly but effectively) 
avoids merging- two components. 






It in easy to see that a component within a hole will vanish (and be counted) 
just in time to allow the surrounding component to collapse down r We do 
not know any equivalent process lor three dimensions. (Consider knots E) 

We believe that further investigations of this sort should lead to a deeper 
understanding of parallel computation in general and of the potential role ol 
such processes in practical artificial and biological visual systems* 

D. Designing a Stereo Vision. Astern 

Another example of computational geometry is the reconstruction of a three- 
dimensional scene from stereo views* In very simple cases - for example, 

where the scene consists of a single point or Jine segment - the problem 
belongs to perspective geometry. But scenes such as those shown In dis- 
cussing SEE raise different problems, One possible approach is to make 
an analysis; using SEE on each stereo view and then mateh identified ob- 
jects. Another "pure* approach is to use the stereo match as a means of 
object identification. Jn practice, a real vision system would try to use a 
mixture Of both. In this ease a new kind of difficulty arises.; that ol mani- 
pulating data of varied kinds and varied degrees of reliability , 

David Perkins has been developing for his Ph + D f thesis a programming; sys- 
tem for handling complex ncL-lik£ data structures thai could allow informa- 
tion of diverse sorts to he built up in the course of exploring a scene. The 
stereo problem is being used as the subject-matter for the system, 

E. Theorem- Proving Programs 

We are concerned about adapting automatic heuristic deductive programs to 
problems with large an J diverse data bases. Gerald Sus.&man has programmed 
a classical theorem -prover based on the resolution principle and has begun 
to gain experience with the Introduction of heuristics of a more specific sort 
than are usually employed, Carl Hewitt, at the other extreme, has developed 
a system and; language called PLANNER to permit the most, diverse possible 
operations that might be used in proofs. Neither project is yet sufficiently 
advanced to warrant detailed a]ialysis uf results. 



55 



Chess and Game- Trees - Richard Greenblatt 

During the year, substantial progress was made on Mechanical Chess - a 
domain thai has long been of great interest to workers in Artificial Intelli- 
gence. The program, known as MACHAC VI, has achieved the level of ability 
of an average amateur player and has been accepted as an honorary mem- 
ber of the American Chess Federation. The clear superiority of this pro- 
gram over all previous ones is interesting both theoretically and as & 
visible demonstration of programming techniques and systems {See Al 
Memo 174). 

For a Ions time we have hoped to see a clarification of the theory of non- 
terminal Evaluation functions in game-playing, tree-search programs. Gne f s 
fir&t inclination is to try to interpret the E- function as an approximation 
to an estimate of probability of success, improved by the Look- ahead pro- 
cedure. No one has been particularly successful at making this interpreta- 
tion workable. Some promising results on probabilistic strategy theory are 
given in a recent thesis by David Johnson. 

Mathema tical Laborator y - William A, Martin and Joel Moses 

It has been our long-term goal to develop a man-machine system for ampli- 
fying the effectiveness of a mathematician working on either applied or ab- 
stract theoretical problems, This project i* proceeding in several directions, 

to bring together such systems as the previously reported Integ ration system 
of J r Moses, the S ymbolic Mathematical Laborator y of W. Martin, and the 
MATHLAB of C. Engelman. A two or three -year project is planned, leading 
to an on-line system to aid In carrying out compter and tedious symbolic 
calculations, and to build up an active store of knowledge about mathematical 
algorithms and heurlsiic methods. We are preparing a monograph on sym- 
bolic algebraic manipulation*!. Martin is pursuing the theory of parsing 
context-free cxjir«HHioii3 t both as pyre theory and in connection with an on- 
line parser for hand-written mathematical formulae. Moses is developing a 
simplification system of advanced power that will, for example, be able to 
convert 

1 + 2 log ( &ln g 2y + x + cos E 2y ) t log^l -i-x) 
log(l + *J + 1 
to 

1 + log { 1 t X ) t 

With this, it should be feasible to loiplement a powerful decision procedure 
far integration due to Risen; the system should be able to integral e 

, n — lOgZ + -+ 1 

+ 



flog ? z + z) 2 Ics^r. + z 



bti 



la 



"^ E + log(log a z + s) 



log E z + z 



A program, called SARGE, which drills students in freshman calculus, inte- 
gration problems lias alto been written by Moses. Students are supposed to 
type step- by-step solution to an integration problem. The computer checks 
the correctness of the justification (e.g., substitution o£ variables) given for 
each step. Sometimes, though rarely, the computer can give a reasonable 
description of the type of mistake the student made in an erroneous step. 

Int eractive Com puter- Mediated Anim ation - Ronald M, Baecker 

Standard techniques of man -machine graphical communication have been sup- 
plemented with the capability for immediate viewing of a synthesized anima- 
tion sequence. These were constructed using three special-purpose animation 
systems which have been implemented with the interactive graphics capability 
of the Lincoln Laboratory TX-2 computer. 

The concept of a table- driven algorithmic synthesis of an animated display 
has been developed as an approach to the specification of picture dynamics. 
The Latnes, called movem ent descriptions , abstract aspects of behavior that 
recur over extended intervals of time in a particular animated display. 
Rh ythm descri ptions express patterns of the triggering, pacing, coordination, 
and synchronizing of picture change. 

Techniques of control that particularly exploir the interactive environment 
have been developed. The animator is coupled to the film under construction, 
both through the sketching and graphical editing of static pictures of dynamic 
behavior (expressed by movement descriptions), and through the dynamic 
mimicking oi desired behavior. The animator's dynamics are expressed 
through stylus, push-buttons, and other devices; the language may be used 
to construct animated visual displays, to build system tools that aid the 
construction process* and to implement special-purpose animation systems. 

Computer time and support fur this research have also been provided by 
the M.LT. Lincoln Laboratory, under a contract from the Advanced Research 
Projects Agency. 

Fourier Trans f orm Methods in Imag e Processin g - BerHiold K* P. Horn 

The two -dimensional Fourier transform has been used widely in optics lor 
the evaluation of image -forming systems, yet has been used infrequently in 
pattern recognition and perception studies , R was thus of interest to find 
areas of image processing to which the Fourier transform could profitably 
be applied. Although the Fourier transform has many useful properties that 
indicate its use in image processing (such as translational invariants) many 
of these arc shared by other functions. 



Si 



The Fourier transform was found to be useful dither in spectral, analyst 
of small image areas, or as a global image -trans former inj 

\) focusing an optical device, 

2) restoring a degraded picture, 

3 J resolution beyond the usual limits, 

4) edge detection in a preprocessing pass,. 

5) confirmation of edges m a j*iven position, 

6) least -squares filtering in the presence of noise, 

7) dyna-mic range reduction. 

Ne, conclusions have teen reached about the usefulness of Fourier methods 
In texture description or curvature detection. A computer program was 
developed for the PDP-6 to allow experimentation on images read in through 
the on-line computer "eye fl , and the Fa3t Fourier Transform method was 
USOd in this program. 

Structure of Atona l Music - Allen Forte 

Some aspect* of this project, which is concerned with a general structural 
description of so-called atonal music, have been presented in MAC Progress 
Hcport m and, mora extensively * in MAC-TR-39. During the past year the 
score-reading program h which parses an input string representing standard 
music notation, was improved in the direction of greater generality and ef- 
ficiency, and more effective tools were developed to deal with certain basic 
problems of musical analysis, Jn particular, the notion of time- point states 
has led to a solution to the problem of segmentation {the determination of 
structural units) and has suggested some useful ways in which relations be- 
tween such unite might be displayed. The task of constructing an appropriate 
model for this music remains difficult. Work done thus far has suggested 
several possibilities which are currently being programmed. 



